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ABSTRACT: In comparison with engineering, engineering technology is more practical and purposeful. 
The engineering technology education programs in Taiwan have been mainly offered in 56 
universities/colleges of technology (UTs/CTs) and are anticipated to continuously improve their 
performance to prepare quality engineering technologists. However, it is necessary to construct well- 
structured and up-to-date performance indicators for the programs. The authors completed a literature 
review and identified 22 key performance indicators (KPIs), which are outcome-based and compatible 
with ABET’s accreditation criteria. Then a questionnaire including the KPIs was mailed to 160 
randomly sampled engineering technology department chairs in the UTs/CTs in Taiwan to solicit their 
opinions. This paper presents the background, process and results of this research study. 



1 ENGINEERING TECHNOLOGY EDUCATION IS HIGHLY EXPECTED TO PERFORM 

WELL 

Beyond nine years of compulsory education, formal education in Taiwan is streamed into the 
following two tracks, which are like “two legs walking” to prepare the workforce: (1) General academic 
education (GAE) — mainly including three years of college-bound coursework in senior high schools 
(SHSs) as well as comprehensive high schools (CHSs), and four to seven years of coursework at the 
academic university/college level; (2) Technological and vocational education (TVE) — mainly including 
the institutions/programs highlighted in Figure 1. All universities/colleges and JCTs in both GCE and 
TVE systems, shown in Figure 1, are categorized as higher education institutions. In the past decade 
(school years 1993-2002), the number of these institutions increased by 29 (or 23.2%; i.e., from 125 to 
154) (see Figure 2). 
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Figure 1 - Formal education structure in Taiwan 
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Figure 2 - The number changes of higher education institutions 

Performance has been defined as “the results of activities of an organization or investment over a 
given period of time” (INVESTWORDS.COM, 1997-2004). All higher education institutions in Taiwan 
are expected to perform well to parallel the increase in their number. In addition, the intense focus placed 
on the employment and innovation of technology in Taiwan has caused a significant need for higher 
education institutions to produce knowledge workers across the broad spectrum of technology. 

Engineering technology (ET) is “the profession in which knowledge of the applied mathematical and 
natural sciences gained by higher education, experience, and practice is devoted to application of 
engineering principles and the implementation of technological advances for the benefit of 
humanity”. Engineering technology education (ETE) for the professional focuses primarily on “analyzing, 
applying, implementing and improving existing technologies, and is aimed at preparing graduates for the 
practice of engineering closest to the product improvement, manufacturing, and engineering operational 
functions” (The ETD of ASEE). 

As a realm of higher education, ETE in Taiwan is more practical and purposeful than engineering 
education (EE) and is mainly offered in UTs/CTs baccalaureate undergraduate programs, and in JCTs 
diploma programs. It is highly expected to perform well to prepare quality engineering technologists and 
technicians. For example, the accreditation system of the Accreditation Board for Engineering and 
Technology (ABET) is being introduced into Taiwan to assure quality in EE and ETE institutions and 
programs, leading to improved performance. However, an ETE program meets ABET’s criteria and 
attains accreditation is only considered as being able to stably “stand on the ground”. It is further 
expected to “reach the sky” by performing in excellence. In other words, accreditation criteria such as 
ABET’s are often seen as “ground level” criteria while criteria such as those in U.S. News & World 
Report magazine's annual survey of national universities are considered as “sky level” criteria. 

2 TWENTY-TWO KPI’S ARE IDENTIFIED FOR ETE PROGRAMS 

Indicators are defined as "individual or composite statistics that reflect important features of a system, 
such as education, health, or economy" (Darling-Hammond, 1992, p.236). Also known as key success 
indicators (KSIs), key performance indicators (KPIs) help an organization define and measure progress 
toward organizational goals. Once an organization has analyzed its mission, identified all its stakeholders, 
and defined its goals, it needs a way to measure progress toward those goals. KPIs are those quantifiable 
measurements (REH, 2004). In other words, KPIs are high-level snapshots of an organization based on 
specific predefined variables (INFORMATION BUILDERS). 

Because the number of JCTs is declining, the ETE programs in Taiwan have mainly been offered in 
56 universities/colleges of technology and elsewhere for the purpose of preparing engineering 
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technologists. It is necessary to construct well-structured and up-to-date KPIs to guide the operations of 
these programs. 

Performance indicators have grown in popularity in higher education organizations around the world. 
For example, the Oregon University System (OUS) in the United States implemented performance 
indicators in the late 1990’s. The OUSs 12 KPIs are as follows: total credit enrolment, new undergraduate 
enrolment, freshman persistence, six-year graduation rates, total degree production, degrees in shortages 
areas, philanthropy, recent graduates, graduate success, faculty compensation, research and development 
(R&D), as well as internships (NPG, 2003). The IPO model is widely suggested to identify program 
performance indicators. The model contains: 

Input (I) — A variety of resources available to the program. 

Process (P) — A set of sub-systems, such as curriculum and instruction, that create the program. 

Output (O) — Consequences of the program. 

Performance indicators identified should be scientific and practical. Based on an extensive review of 
literature, the authors identified 22 KPIs for ETE programs which meet the following criteria 
(SHAVELSON, MCDONNELL & OAKES, 1991): 

1. Alignment with UT’s/CT’s missions 

2. Linkage to overall ETE program objectives 

3. Access to specific information regarding program problems 

4. Measurement of observed behavior rather than perceptions 

5. Reliability and validity 

6. Access to analytical links 

7. Feasibility of implementation 

8. Addresses a broad range of audiences 

3 NINETY-ONE ETE DEPARTMENT CHAIRS REVIEWED THE 22 KPIS 

In order to further solicit more opinions on the 22 KPIs identified and shown in Table 1, a 
questionnaire was developed and mailed to request reviews from ETE department chairs. The target 
population of this questionnaire survey was all 273 ETE department chairs in 56 UTs/CTs. Based upon 
the stratification of institutional categories (public/private), 160 sample ETE department chairs were 
randomly selected. The confidence interval of the sample size is 5%. 

At the beginning of March 2004, the coded questionnaire was mailed to 160 sample department 
chairs, who were requested to rate the importance of each KPI on a five-point scale (1-5, in ascending 
order). About 10 days after the initial mailing, a follow-up was e-mailed to all non-respondents. Finally, 
91 (or 56.8%) department chairs responded. 



Table 1. The 22 KPIs Identified and Reviewed by 91 ETE Department Chairs 



KPI 


N 


M 


Rank 


SD 


t 


1. Number of admitted freshman 


90 


3.8 


9 


1.09 


6.661* 


2. Percentage of freshmen to all admission applicants 


90 


3.6 


12 


1.11 


MB 


3. Percentage of enrolled freshman to admitted freshman 


91 


3.9 


6 


1.14 




4. Percentage of freshman who advance to become sophomores 


89 


3.7 


11 


1.03 


EEEEfB 


5. Percentage of graduates from the previous school year to the 
quantity of the initial enrolment 


81 


3.5 


14 


.95 


m 


6. Percentage of graduates of the previous school year who 
participated in work-based practicum 


84 


3.0 


22 


1.11 


-.394 


7. Cooperative institutions’ satisfaction with students participating 
in work-based practicum 


80 


3.5 


14 


1.10 




8. Average graduation credits earned by graduates of the previous 
school year 


85 


3.3 


19 


.95 




9. Percentage of selective credits to average graduation credits 
earned by graduates of the previous school year 


85 


3.1 


21 


.91 


1.423 
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10. Average academic grading of graduates of the previous school 
year 


85 


3.3 


19 


.87 




11. Average behavioural conduct grading of graduates of the 
previous school year 


85 


3.4 


16 


.99 




12. Percentage of graduates of the previous school year employed 
within three months after graduation 


85 


4.0 


3 


1.10 


£ 

8.115 


13. Employed graduates’ satisfaction with their ETE learning at six 
months after graduation 


83 


4.0 


3 


.88 


BH 


14. Employed graduates’ satisfaction with their compensation at six 
months after graduation 


84 


3.6 


12 


.83 




15. Employers satisfaction with ETE graduates 


83 


4.3 


2 


.82 




16. Percentage of graduates of the previous school year 
immediately continuing postgraduate education 


86 


3.3 


19 


.94 


m 


17. Continuing study graduates’ satisfaction with their ETE 
learning at six months after graduation 


85 


3.9 


6 


.87 


9.849* 


18. Continuing study institutions’ satisfaction with ETE graduates 


84 


4.0 


3 


.88 




19. Ratio of students to full-time faculty 


89 


3.9 


6 


1.07 


8.323* 


20. Current revenues per ETE student in the previous fiscal year 


88 


3.8 


9 


.98 


7.580* 


21. Capital revenues per ETE student in the previous fiscal year 


88 


3.9 


6 


1.00 


HEm 


22. Rating from the recent evaluation administered by the Ministry 
of Education (MOE) 


84 




1 


.82 





Note: 1. Most KPIs did not receive all 91 chairs’ ratings, so the number of respondents (N) for each KPI 
may be different. 

2. * p < .05. 



4 CONCLUSIONS 

KPI is a measurable factor of extreme importance to the program in achieving its strategic 
goals and in presenting its effectiveness. Based on the 22 KPIs identified and reviewed, the 
following three conclusions may be drawn: 

1. 20 KPIs should be highly valued. 

As shown in Table 1, the means ( M) of 20 KPIs (excluding 6 and 9) are statistically 
significantly higher than the theoretical average of 3.0. That is to say, the 20 KPIs can 
be highly recommended to all ETE programs. 

2. The 20 KPIs can be prioritized according to their ranks. 

The UTs/CTs in Taiwan may or may not simultaneously address all 20 KPIs at the initial 
stage. It is suggested that the 20 KPIs can be prioritized according to their ranks shown 
in Table 1. In other words, the KPIs rated in higher ranks should get higher priorities. In 
addition, more appropriate KPIs may be further identified and added on. 

3. The 19 KPIs (excluding 6, 9, and 22) should be infused into the evaluation administered 
by the MOE. 

The ETE programs and their host UTs/CTs should continuously manage and improve 
their own performance. However, the CTs receive an outside evaluation administered 
by the MOE every three years. As shown in Table 1, the evaluation results are highly 
valued by the ETE department chairs. Therefore, the 19 KPIs should be appropriately 
infused into the routine evaluation of CTs. 
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